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To  the  Reader 


This  document  provides  a  high-level  overview  of  the  CMM^’^-Based  Appraisal^  for  Internal 
Process  Improvement  (CBA  IPI)  V1.1  assessment  method.  It  is  primarily  intended  for  a 
sponsor  or  an  opinion  leader  in  an  organization  who  is  considering  an  assessment.  Additional 
audiences  for  the  document  include  potential  team  members,  assessment  participants,  and 
individuals  who  are  involved  in  or  may  be  interested  in  process  improvement. 

The  document  addresses  things  that  must  be  considered  when  planning  a  CBA  IPI.  Also  dis¬ 
cussed  are  software  process  improvement  benefits  that  organizations  have  realized  where  as¬ 
sessments  were  key  components.  In  addition,  resources  required  to  conduct  a  CBA  IPI  are 
discussed.  This  document  is  intended  to  provide  an  overview  of  the  method,  not  specific  infor¬ 
mation  on  how  to  conduct  a  CBA  IPI.  Persons  wishing  to  conduct  a  CBA  IPI  in  their  organiza¬ 
tion  should  contact  the  SEI  to  ensure  that  the  assessment  is  performed  by  an  authorized  Lead 
Assessor.  Appendix  A  describes  the  experience,  training,  and  qualifications  required  of  Lead 
Assessors. 


1.  sMQgpgpjijty  Matunty  Model,  CMM,  and  IDEAL  are  service  marks  of  Carnegie  Mel¬ 
lon  University. 
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CMM^'^-Based  Appraisal  for  Internal 
Process  Improvement  (CBA  IPI): 
Method  Description 

Abstract:  This  document  is  a  high-level  overview  of  the  CMM^'^-Based 
Appraisal  for -Internal  Process  Improvement  (CBA  IPI)  VI. 1  assessment 
method.  It  provides  a  brief  history  of  SEI  appraisal  methods,  as  well  as 
establishing  appraisals  in  the  context  of  the  IDEAL®'^  approach  to  software 
process  improvement.  CBA  IPI  is  a  diagnostic  tool  that  supports,  enables,  and 
encourages  an  organization’s  commitment  to  process  improvement.  The 
method  helps  an  organization  gain  insight  into  its  software  development 
capability  by  identifying  strengths  and  weaknesses  of  its  current  processes 
related  to  the  Capability  Maturity  Model^'^for  Software  VI. 1.  The  method 
focuses  on  identifying  software  improvements  that  are  most  beneficial,  given 
an  organization’s  business  goals  and  current  maturity  level.  Brief  descriptions 
of  the  method  activities,  roles,  and  responsibilities  are  provided.  In  addition, 
guidelines  are  provided  for  establishing  resource  requirements  for  conducting 
a  CBA  IPI.  The  SEI  Appraiser  Program  is  discussed,  detailing  the  requirements 
for  persons  qualified  to  lead  CBA  I  Pis. 


1  Why  Was  CBA  IPI  Created? 

Using  the  Capability  Maturity  Model®''^  for  Software  V.1 .1  (CMM)  as  a  reference  model  [Paulk 
93a,  Paulk  93b],  the  Software  Engineering  Institute  (SEI)  developed  the  CBA  IPI  V1 .0  In  1 995 
for  assessing  an  organization’s  software  process  capability.  CBA  IPI  V1.1  was  released  in 
1996  containing  modifications  for  clarification  and  simplification.  In-depth  documentation  and 
guidance  on  the  CBA  IPI  method  [Dunaway  96]  is  available  through  CBA  Lead  Assessor 
Training. 

1 .1  History  of  SEI  Appraisal  Methods 

In  accordance  with  the  SEI’s  mission  to  provide  leadership  in  advancing  the  state  of  the  prac¬ 
tice  of  software  engineering  by  improving  the  quality  of  systems  that  depend  on  software, 
there  has  been  strong  emphasis  within  the  SEI  on  treating  software  development  tasks  as  pro¬ 
cesses  that  can  be  defined,  practiced,  measured,  and  improved.  In  early  software  process 
publications,  a  software  maturity  framework  [Humphrey  87a]  and  questionnaire  [Humphrey 
87b]  were  developed  to  help  organizations  characterize  the  current  state  of  their  software 
practices,  set  goals  for  process  improvement,  and  set  priorities. 

Software  process  is  defined  to  mean  the  system  of  all  tasks  and  the  supporting  tools, 
standards,  methods,  and  practices  involved  in  the  production  and  evolution  of  a  software 
product  throughout  the  software  life  cycle.  It  has  become  widely  accepted  that  the  quality  of  a 
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(software)  system  is  largely  governed  by  the  quality  of  the  process  used  to  develop  and 
maintain  it  [Crosby  79,  Deming  86,  Humphrey  90,  Juran  88]. 

The  SEI  assisted  a  number  of  organizations  in  performing  assessments  [Olson  89]  based 
largely  on  the  maturity  questionnaire.  This  early  questionnaire  provided  a  scoring  mechanism 
for  determining  a  project’s  maturity  level.  In  1988-91,  the  SEI  provided  training  to  organiza¬ 
tions  who  wished  to  perform  self-assessments  of  their  software  processes. 

In  1 990  the  SEI  commercialized  the  software  process  assessment  (SPA)  to  more  broadly  dis¬ 
seminate  the  technology,  since  the  SEI  was  not  equipped  to  handle  the  demand  for  assess¬ 
ment  services.  Industry  and  government  licensees  were  selected  as  vendors  to  market 
assessment  services.  During  1991-1993,  SEI  self-assessment  training  was  gradually  phased 
out  and  replaced  by  vendor  training.  Data  from  these  assessments  have  been  collected  by  the 
SEI,  and  periodic  reports  are  delivered  on  the  state  of  the  practice.  The  initial  state  of  the  prac¬ 
tice  reported  on  assessment  data  largely  from  project-based  questionnaire  data  from  10  sites 
[Humphrey  89].  The  follow-up  report  included  the  first  site-based  software  process  maturity 
profile  and  provided  an  analysis  of  assessment  findings  from  59  sites  showing  the  frequency 
with  which  key  process  deficiencies  were  identified  by  assessment  teams  [Kitson  92].  A  recent 
report  presents  an  analysis  of  assessment  results  from  48  organizations  that  have  performed 
2  or  more  assessments  and  focuses  on  the  time  required  to  increase  process  maturity  [Hayes 
95].  Updates  on  the  process  maturity  profile  of  the  software  community  are  published  twice  a 
year  with  the  latest  profile  containing  data  on  515  assessments  representing  440  distinct  sites 
[Zubrow  95]. 

Based  on  the  success  and  widespread  use  of  the  maturity  framework  and  software  process 
assessments,  Version  1.0  of  the  CMM  for  Software  was  published  in  1991  [Paulk  91a,  Paulk 
91b].  In  1993  the  CMM  was  revised,  and  Version  1.1  was  published.  Various  organizations 
modified  SEI  appraisals  to  reflect  the  CMM;  however,  the  CBA  IPI  method  is  the  first  CMM- 
based  assessment  method  released  by  the  SEI.  CBA  IPI  uses  an  updated  maturity  question¬ 
naire  consistent  with  CMM  VI. 1.  Unlike  the  maturity  questionnaire  released  in  1987,  the  cur¬ 
rent  maturity  questionnaire  does  not  include  a  mechanism  for  scoring  maturity  levels  [Zubrow 
94]. 

The  SEI  has  published  reports  that  show  a  relationship  between  CMM-based  improvement 
and  organizational  performance.  One  report  [Herbsleb  94]  documents  process  improvement 
efforts  in  1 3  organizations  by  showing  improvements  in  cycle  time,  defect  density,  and  produc¬ 
tivity.  Benefit-to-cost  ratios  presented  range  from  4.0:1  to  8.8:1.  In  a  more  comprehensive  re¬ 
port  on  the  impact  of  CMM-based  appraisals  on  subsequent  software  process  improvement 
and  organizational  performance  [Goldenson  95],  survey  results  from  138  appraisal  partici¬ 
pants  representing  56  appraisals  are  presented.  The  results  show  that,  in  general,  increased 
process  maturity  results  in  better  product  quality,  ability  to  meet  schedule  commitments,  and 
other  indicators  of  organizational  performance. 
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1 .2  IDEAL^''^’  Approach  to  Process  Improvement 

The  CBA  IPI  method  is  a  diagnostic  tool  used  to  appraise  and  characterize  an  organization’s 
software  processes  and  is  used  in  the  larger  context  of  software  process  improvement.The 
SEI’s  recommended  framework  for  software  process  improvement  is  the  IDEAL®*'^  model, 
shown  in  Figure  1  [Peterson  95,  Radice  94].  The  IDEAL  approach  consists  of  five  phases:  ini¬ 
tiating,  diagnosing,  acting,  establishing,  and  leveraging.  Appraisals  are  an  integral  part  of  the 
diagnosing  phase  of  the  IDEAL  approach. 


Figure  1 :  IDEAL®"'^  Model  for  Software  Process  Improvement 


The  initiating  phase  of  process  improvement  should  be  successfully  undertaken  before  the  ap¬ 
praisal  start-up.  First,  some  outside  stimulus  for  improvement  needs  to  occur.  Then  sponsor¬ 
ship  is  established  with  the  site’s  senior  management,  and  efforts  toward  building  an 
improvement  infrastructure  are  committed.  An  organization  could  be  appraised  for  the  first 
time  or  could  be  reappraising  their  processes  in  preparation  for  the  next  process  improvement 
cycle.  The  IDEAL  approach  assumes  that  the  appraisal  will  be  followed  by  a  thorough  docu¬ 
mentation  of  the  appraisal  results  and  the  development  of  recommendations.  The  software 
process  baseline  established  by  the  appraisal  and  the  recommendations  and  priorities  that 
come  from  the  appraisal  form  a  basis  for  follow-on  activities.  These  activities  are  typically  doc¬ 
umented  in  an  action  plan  for  furthering  process  improvement  activities. 

1 .3  Current  SEI  Appraisal  Methods 

To  provide  a  framework  for  rating  an  organization’s  process  capability  against  the  CMM  and 
to  provide  a  basis  for  comparing  assessment  and  evaluation  results,  the  SEI  published  the 
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CMM  Appraisal  Framework  (CAF)  [Masters  95].  The  CAF  identifies  the  requirements  and  de¬ 
sired  characteristics  of  a  CMM-based  appraisal  method  in  order  to  improve  consistency  and 
reliability  of  methods  and  their  results.  An  appraisal  method  is  CAF  compliant  when  it  meets 
all  of  the  CAF  requirements.  The  term  appraisal  as  used  at  the  SEI  includes  multiple  methods, 
such  as  assessments  and  evaluations,  all  of  which  focus  on  an  organization’s  software  devel¬ 
opment  process. 

Both  CBA  IPI  and  Software  Capability  Evaluation-.(SCE)  V3.0  were  designed  to  be  CAF  com¬ 
pliant.  The  following  subsections  briefly  describe  the  primary  SEI  appraisal  methods,  CBA  IPI 
and  SCE,  and  the  differences  between  them.  Interim  Profile  [Whitney  94]  is  a  method  to  rap¬ 
idly  measure  the  status  of  an  organization’s  software  engineering  process  improvements  be¬ 
tween  organizational  assessments  and  was  not  intended  to  comply  with  the  CAF;  therefore,  it 
is  not  included  in  our  discussion  of  current  SEI  appraisal  methods. 

1 .3.1  CMM-Based  Appraisal  for  Internal  Process  Improvement  (CBA  IPI) 

The  CBA  IPI  method  was  created  in  response  to  user  needs  for  a  CMM-based  assessment 
method.  The  SPA  method,  which  has  become  so  familiar  in  the  software  community,  pre-dat- 
ed  the  CMM.  Although  many  organizations  have  modified  the  SPA  to  reflect  the  CMM,  there 
was  a  wide  range  of  approaches  and  results. 

The  CBA  IPI  method  was  developed  and  field  tested  in  1 994.  After  factoring  in  lessons  learned 
from  the  community  feedback,  the  SEI  released  CBA  IPI  V1 .0  in  May  1995.  The  method  and 
documentation  were  upgraded  to  CBA  IPI  VI  .t  in  March  1996.  The  CBA  IPI  method  explicitly 
uses  CMM  VI. 1  as  a  reference  model.  The  data  collection  is  based  on  key  process  areas 
(KPAs)  of  the  CMM  as  well  as  non-CMM  issues.  CBA  IPI  is  intended  to  establish  consistency 
among  CMM-based  assessments  so  that  results  from  one  assessment  can  be  compared  to 
those  of  another.  The  CBA  IPI  method  complies  with  the  CAF,  so  results  from  a  CBA  IPI  are 
intended  to  be  consistent  with  results  from  other  CAF-compliant  methods.  The  CBA  IPI  meth¬ 
od  is  described  in  more  detail  in  the  following  sections  of  this  report.  Appendix  B  contains  de¬ 
scriptions  of  the  method  roles  and  responsibilities,  while  Appendix  C  contains  a  glossary  of 
terms  commonly  used  in  a  CBA  IPI. 

1 .3.2  Software  Capability  Evaluation  (SCE) 

SCEs  are  used  for  software  acquisition  as  a  discriminator  to  select  suppliers,  for  contract 
monitoring,  and  for  incentives.  They  can  also  be  used  for  evaluation  of  internal  processes. 
SCE  V2.0  [CBA  Project  94]  was  updated  to  reflect  CMM  VI  .1 .  SCE  V3.0  [Byrnes  96]  is  CAF 
compliant.  Therefore,  results  from  a  SCE  V3.0  should  be  consistent  with  a  CBA  IPI  VI  .1  if  the 
areas  of  investigation  are  the  same  and  in  relatively  the  same  time  frame.  SCE  is  used  to  gain 
insight  into  the  software  process  capability  of  a  supplier  organization  and  is  intended  to  help 
decision  makers  make  better  acquisition  decisions,  improve  subcontractor  performance,  and 
provide  insight  to  a  purchasing  organization. 
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1.3.3  Differences  Between  Assessments  and  Evaluations 

The  basic  difference  between  an  assessment  and  an  evaluation  is  that  an  assessment  is  an 
appraisal  that  an  organization  does  to  and  for  itself,  and  an  evaluation  is  an  appraisal  where 
an  external  group  comes  into  an  organization  and  looks  at  the  organization’s  process  capabil¬ 
ity  in  order  to  make  a  decision  regarding  future  business.The  scope  of  an  assessment  is  de¬ 
termined  relative  to  the  organization’s  needs  and  the  business  goals  of  the  sponsor,  who  is 
usually  the  senior  manager  of  the  assessed  organization.  In  contrast,  the  scope  of  an  evalu¬ 
ation  is  determined  relative  to  the  needs  of  the  sponsor,  who  is  the  individual  or  individuals 
responsible  for  deciding  to  conduct  the  evaluation  of  the  organization(s)  with  whom  the  spon¬ 
sor  is  currently  or  considering  doing  business. 

After  an  assessment,  the  senior  manager  of  the  assessed  organization  owns  the  assessment 
findings  and  results  and  generally  uses  the  results  to  formulate  an  action  plan  for  the  process 
improvement  program.  After  an  evaluation,  the  sponsor  owns  the  evaluation  results  and  uses 
the  results  to  make  decisions  regarding  the  recipient  organization(s),  such  as  source  selec¬ 
tion,  incentive  fee  awards,  performance  monitoring,  risk  management,  and  measurement  of 
internal  improvement;  these  results  may  or  may  not  be  shared  with  the  evaluated  organization. 

Assessments  are  intended  to  motivate  organizations  to  initiate  or  continue  software  process 
improvement  programs.  Evaluations  are  externally  imposed  motivation  for  organizations  to 
undertake  process  improvement.  Assessment  teams  represent  a  collaboration  among  team 
members,  many  of  whom  are  from  the  organization  being  assessed.  Evaluation  teams,  on  the 
other  hand,  take  more  of  an  audit-oriented  approach  with  a  more  formal  interface  between 
team  members  and  organization  representatives;  evaluation  teams  rarely  include  representa¬ 
tives  from  the  evaluated  organization. 
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2  What  Is  the  CBA  IPI  Method? 


The  CBA  IPI  method  is  a  diagnostic  tool  that  enables  an  organization  to  gain  insight  into  its 
software  development  capability  by  identifying  strengths  and  weaknesses  of  its  current  pro¬ 
cesses,  to  relate  these  strengths  and  weaknesses  to  the  CMM,  to  prioritize  software  improve¬ 
ment  plans,  and  to  focus  on  software  improvements  that  are  most  beneficial,  given  its  current 
level  of  maturity  and  the  business  goals. 

The  method  is  an  assessment  of  an  organization’s  software  process  capability  by  a  trained 
group  of  professionals  who  work  as  a  team  to  generate  findings  and  ratings  relative  to  the 
CMM  key  process  areas  within  the  assessment  scope.  The  findings  are  generated  from  data 
collected  from  questionnaires,  document  review,  presentations,  and  in-depth  interviews  with 
middle  managers,  project  leaders,  and  software  practitioners. 

The  CBA  IPI  method  has  two  primary  goals: 

•  to  support,  enable,  and  encourage  an  organization’s  commitment  to  software 
process  improvement 

•  to  provide  an  accurate  picture  of  the  strengths  and  weaknesses  of  the 
organization’s  current  software  process,  using  the  CMM  as  a  reference 
model,  and  to  identify  key  process  areas  for  improvement. 

The  approach  of  the  CBA  IPI  method  is  to  assemble  and  train  a  competent  assessment  team 
under  the  leadership  of  a  Lead  Assessor  and  to  conduct  a  structured  series  of  activities  with 
key  people  in  the  organization  to  understand  their  problems,  concerns,  and  ideas  for  improve¬ 
ment.  The  method  is  based  on  the  following  key  assessment  principles: 

•  Use  the  Capability  Maturity  Model  for  Software  V1 .1  as  a  process  reference 
model. 

•  Use  a  formalized  assessment  process  that  complies  with  the  CMM  Appraisal 
Framework. 

•  Involve  senior  management  as  the  assessment  sponsor. 

•  Base  the  assessment  on  the  sponsor’s  business  goals  and  needs. 

•  Observe  strict  confidentiality  by  guaranteeing  that  no  information  will  be 
attributed  to  an  individual  or  project. 

•  Approach  the  assessment  as  a  collaboration  between  the  assessment  team 
and  the  organizational  participants. 

2.1  What  Can  This  Method  Accomplish? 

The  business  needs  for  process  improvement  drive  the  requirements  for  an  assessment.  The 
business  needs  for  process  improvement  generally  include  one  or  more  of  three  closely  relat¬ 
ed  factors:  reducing  costs,  improving  quality,  and  decreasing  time  to  market.  The  fundamental 
assumption  is  that  these  factors  are  largely  determined  by  the  development  processes. 
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Since  the  CBA  IPI  method  is  designed  to  support  a  variety  of  assessment  needs,  the  results 
of  a  CBA  IPI  can  support  a  variety  of  activities  that  require  detailed  knowledge  about  the 
strengths  and  weaknesses  of  the  software  process,  such  as 

•  establishing  software  process  action  plans 

•  measuring  actual  progress  against  action  plans 

•  identifying  best  (most  effective)  practices  within  the  organization  for 
transition -elsewhere  in  the  organization 

The  CBA  IPI  method  must  build  on  an  organization’s  commitment  established  during  previous 
phase(s)  of  the  software  process  improvement  cycle.  To  support  the  goal  of  enabling  and  sup¬ 
porting  an  organization’s  commitment  to  process  improvement,  the  assessment  team  and  as¬ 
sessment  participants  must  understand  the  sponsor’s  business  goals  and  allow  for  a 
collaborative  shaping  of  the  assessment  scope.  It  is  important  that  the  organization  owns  and 
“buys  in”  to  the  assessment  results,  identifying  the  most  critical  issues  (CMM  and  non-CMM) 
that  need  to  be  addressed  in  order  to  sustain  momentum  for  the  follow-on  activities. 

The  other  primary  goal  of  the  CBA  IPI  is  to  provide  an  accurate  picture  of  existing  software 
processes.  To  accomplish  this,  the  assessment  team  will 

•  provide  the  organization  with  the  data  needed  to  baseline  the  organization’s 
software  capability 

•  identify  major  non-CMM  issues  that  have  an  impact  on  process  improvement 
efforts 

•  identify  strengths  and  weaknesses  using  the  CMM  as  a  reference  model 

•  provide  findings  to  the  sponsor  and  the  organization  that  are  sufficiently 
complete  to  guide  the  organization  in  planning  and  prioritizing  future  process 
improvement  activities 

2.2  Minimum  Requirements  for  a  CBA  IPI 

For  an  assessment  to  be  considered  a  CBA  IPI,  the  assessment  must  meet  the  following  min¬ 
imum  requirements  concerning  the  assessment  team,  assessment  plan,  data  collection,  data 
validation,  rating,  and  reporting  of  assessment  results.  Permissible  tailoring  options  are  pro¬ 
vided  with  the  requirements. 

2.2.1  Assessment  Team 

The  assessment  team  must  satisfy  the  following  requirements: 

•  The  assessment  team  must  be  led  by  an  authorized  SEI  Lead  Assessor. 

•  The  team  shall  consist  of  a  minimum  of  4  and  a  maximum  of  10  team 
members.  At  least  one  team  member  must  be  from  the  organization  being 
assessed. 
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•  All  team  members  must  receive  the  SEI's  Introduction  to  the  CMM  course,  or 
its  equivalent,  and  the  SEI’s  CBA  IPI  team  training  course. 

•  Team  members  must  meet  the  selection  guidelines  relative  to  software 
engineering  and  management  experience. 

Tailoring  options  are  as  follows: 

•  the  size  of  the  assessment  team,  as  long  as  the  team  consists  of  between  4 
and  10  qualified  individuals 

•  the  composition  of  the  team  as  to  whether  they  are  internal  or  external  to  the 
organization,  as  long  as  one  team  member  is  from  the  organization  being 
assessed 

2.2.2  Assessment  Plan 

An  assessment  plan  must  be  created  that  at  a  minimum  contains  the  following: 

•  the  goals  for  the  assessment 

•  the  CMM  scope  (KPAs  to  be  examined)  and  the  organization  scope  for  the 
assessment  including  selected  projects  and  assessment  participants 

•  a  schedule  for  assessment  activities  and  identification  of  the  resources  to 
perform  the  activities 

•  the  assessment  outputs  and  any  anticipated  follow-on  activities 

•  planned  tailoring  of  the  assessment  method 

•  risks  and  constraints  associated  with  execution  of  the  assessment 

•  the  sponsor’s  authorization  for  the  assessment  to  be  conducted 

Tailoring  options  are  as  follows: 

•  weighting  of  the  assessment  goals.  Depending  upon  the  organization’s 
motivation  for  having  an  assessment,  more  emphasis  may  be  placed  on  one 
goal  than  the  other,  e.g.,  more  focus  toward  supporting  an  organization’s 
software  process  improvement  than  precise  conformance  relative  to  the 
CMM. 

•  the  specific  organizational  entities  that  comprise  the  organization’s  scope  for 
the  assessment 

•  the  specific  KPAs  selected  that  comprise  the  CMM  scope  for  the  assessment 

•  the  number  of  projects  and  their  particular  characteristics 

•  the  length  of  time  for  the  assessment 

2.2.3  Data  Collection 

Assessment  data  must  be  classified  with  respect  to  four  data  collection  categories  (instru¬ 
ments,  presentations,  interviews,  and  documents)  and  at  a  minimum  contain  the  following: 

•  instrument  data  (maturity  questionnaire  responses)  from  at  least  the  project 
leaders  from  the  selected  projects 
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•  interview  data  from  project  ieaders  from  selected  projects  via  individual 
interviews 

•  interview  data  from  functional  area  representatives  (practitioners)  and  middle 
managers  via  group  interviews 

•  document  data  for  each  of  the  KPA  goals  within  the  CMM  scope  of  the 
assessment 

•  presentation  data  via  a  review  of  the  draft  findings  with  the  assessment 
participants 

In  addition,  confidentiality  of  data  sources  must  be  protected. 

Tailoring  options  are  as  follows: 

•  Collect  instrument  data  from  more  respondents  than  the  project  leaders. 

•  Collect  the  site  information  packet. 

•  Conduct  a  project  leader  interview  with  more  than  one  project  representative. 

•  Conduct  part  of  a  group  interview  “free  form”  where  interviewees  are  asked 
to  discuss  anything  they  feel  the  assessment  team  should  know. 

•  Increase  the  emphasis  on  collecting  document  data. 

•  Vary  the  number  of  draft  finding  sessions  that  are  held. 

2.2.4  Data  Validation 

Data  must  be  validated  using  the  rules  of  corroboration  and  must  sufficiently  cover  the  CMM 
components  within  the  assessment  scope,  the  organization,  and  the  software  development 
life  cycle. 

The  rules  of  corroboration  are  as  follows: 

•  Observations^  are  based  on  data  from  at  least  two  independent  sources, 
e.g.,  two  separate  people  or  a  person  and  a  document. 

•  Obsen/ations  are  based  on  data  obtained  during  at  least  two  different  data 
gathering  sessions. 

•  Observations  are  confirmed  by  at  least  one  data  source  reflecting  work 
actually  being  done,  e.g.,  an  implementation  level  document  or  an  interview 
with  a  person  who  is  performing  the  work. 

Tailoring  options  are  as  follows: 

•  use  of  individuals  or  mini-teams  for  data  gathering  and  consolidation  tasks 

•  extent  of  documentation  that  is  collected 


1 .  Observations  are  based  on  information  extracted  from  data  collection  sessions. 

To 
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2.2.5  Rating 

Ratings  must  be  based  on  the  CAP  criteria  for  rating  the  process  maturity  of  an  organization 
against  the  CMM. 

Tailoring  options  are  as  follows: 

•  Add  a  “partially  satisfied”  rating  which  would  translate  to  “unsatisfied”  for 
maturity  level  rating. 

•  Extend  ratings  to  common  features  and/or  key  practices. 

•  Rate  the  organization's  maturity  level. 

2.2.6  Reporting  of  Assessment  Results 

A  final  findings  briefing  must  be  given  to  the  sponsor  that  presents  the  strengths  and  weak¬ 
nesses  of  each  KPA  within  the  assessment  scope,  as  well  as  a  KPA  profile  that  indicates 
whether  KPAs  are  satisfied,  unsatisfied,  not  rated,  or  not  applicable.  These  data  must  be  re¬ 
ported  back  to  the  SEI. 

Tailoring  options  are  as  follows: 

•  Include  consequences  and/or  recommendations  with  the  assessment 
findings. 

•  Generate  a  written  final  assessment  report  that  details  the  assessment 
findings. 

•  Produce  project-specific  reports  (this  will  require  modification  of  the 
confidentiality  agreement). 
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3  Who  Participates  in  a  CBA  IPI? 


In  order  to  ensure  that  a  CBA  IPI  has  a  successful  impact  on  the  assessed  organization,  many 
factors  need  careful  consideration.  Foremost,  the  person  who  leads  the  assessment  team 
must  be  well  qualified  and  trained.  In  addition,  members  of  the  assessment  team  must  meet 
certain  criteria  for  the  team  composition.  Furthermore,  the  selection  of  the  assessment 
participants  must  adequately  cover  the  scope  of  the  assessed  organization. 


3.1  Who  Is  Qualified  to  Lead  a  CBA  IPI? 

A  CBA  IPI  must  be  led  by  an  authorized  Lead  Assessor  in  the  SEI  Appraiser  Program.  Expe¬ 
rience  in  performing  various  assessments  has  proven  that  it  is  critical  for  assessments  to  be 
led  by  qualified  and  well-trained  individuals  in  order  to  achieve  accurate  results  in  a  reason¬ 
able  amount  of  time  and  achieve  a  successful  organization  intervention.  These  individuals 
must  have  software  development  experience,  CMM  knowledge  and  experience,  and  training 
in  assessment  technology. 

By  having  well-trained  and  qualified  assessors,  different  assessors  should  get  similar  results 
in  organizations  of  similar  maturity.  The  SEI  Appraiser  Program  is  designed  to  maintain  the 
quality  of  appraisal  leaders  within  the  software  community.  The  SEI  strives  to  ensure  the 
continued  confidence  of  the  U.S.  software  community  in  the  quality  of  SEI  appraisal 
technologies.  Lead  Assessors  are  authorized  within  the  program  to  perform  CBA  IPIs,  using 
SEI  copyrighted  materials.  Lead  Assessors  are  authorized  to  market  and  perform 
assessments  either  for  third-party  organizations  or  for  their  own  organizations’  internal 
use.The  SEI  publishes  the  SE/ Appra/ser  D/recto/y  semi-annually  listing  the  authorized  Lead 
Assessors.  A  copy  of  this  directory  and  the  detailed  requirements  for  a  person  to  qualify  as  a 
Lead  Assessor  are  available  from  the  SEI  Customer  Relations  Office  and  through  the  SEI’s 
world  wide  web  page  (http://www.sei.cmu.edu).  Appendix  A  describes  the  SEI  Appraiser 
Program. 

3.2  What  Are  the  Requirements  for  Assessment  Team  Members? 

The  size  and  qualifications  of  the  assessment  team,  along  with  any  potential  biases  of  site 
team  members,  may  affect  the  confidence  the  sponsor  has  in  the  assessment  results.  This  is 
particularly  true  if  the  team  member  has  been  responsible  for  implementing  process  improve¬ 
ments. 

Selection  of  the  assessment  team  takes  into  account  the  knowledge,  skills,  and  abilities  of  the 
assessment  team  as  a  whole  as  well  as  each  individual  team  member.  The  team  as  a  whole 
must  have  the  collective  knowledge,  skills,  and  ability  to  conduct  a  CBA  IPI  assessment  for 
the  particular  organization  being  assessed.  Team  members  are  selected  so  that  their  com¬ 
bined  experience  and  skills  match  what  is  required  for  this  assessment.  Note  that  an  assess- 
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ment  team  size  of  8  individuals  is  recommended  (not  less  than  4  nor  more  than  1 0)  for  several 
reasons.  Smaller  teams  may  have  difficulty  performing  extensive  document  review  and 
achieving  accuracy  in  interviewing  activities.  Larger  teams  require  more  coordination  and  time 
in  order  to  come  to  consensus  on  the  judgments  that  need  to  be  made  by  the  team  as  a  whole. 

Factors  to  be  considered  in  selecting  assessment  team  members  include  the  following: 

•  knowledge  of  the  CMM.  All  team  members  must  have  successfully 
completed  CMM  training  prior  to  CBA  IPi  team  training.  Team  members  must 
have  knowledge  of  each  of  the  KPAs  within  the  anticipated  maturity  level  and 
lower  maturity  levels.  This  knowledge  includes  the  ability  to  explain  the  KPA 
and  its  intent,  and  to  provide  examples  relevant  to  the  appraised  entity. 

•  knowledge  of  process  assessments.  All  team  members  must  complete  CBA 
IPI  team  training  prior  to  the  assessment.  Team  members  with  previous 
experience  in  assessments  will  provide  additional  strength  to  the  team. 

•  knowledge  of  software  process  improvement  concepts.  All  team  members 
must  be  knowledgeable  in  software  process  improvement  concepts. 

•  software  engineering  field  experience.  The  team  must  have  a  minimum  of  25 
years  of  combined  field  experience.  The  average  experience  for  individual 
team  members  must  be  at  least  six  years,  with  no  team  member  having  less 
than  three  years  of  software  field  experience. 

•  management  experience.  The  team  must  have  a  minimum  of  10  years  of 
combined  management  experience,  and  at  least  one  team  member  must 
have  6  years  of  management  experience. 

•  life-cycle  phases  and  functional  activities  experience.  Seventy-five  percent  of 
the  team  members  must  have  experience  in  at  least  one-third  of  the 
organization’s  software  life-cycle  phases  and  their  associated  functional 
activities.  At  least  two  team  members  must  have  had  experience  in  each  life- 
cycle  activity. 

•  organizational  environment,  applications,  and  existing  software  process 
knowledge.  At  least  one  team  member  must  be  knowledgeable  in  both  the 
organization’s  environment  and  application  domain.  Site  team  members  are 
important  to  the  team;  however,  these  team  members  must  not  have  a 
vested  interest  in  the  assessment  results. 

•  team  skills.  Each  team  member  must  have  good  written  and  oral 
communication  skills,  the  ability  to  facilitate  free  flow  of  information,  and  the 
ability  to  perform  as  team  players  and  negotiate  consensus. 

•  credibility.  Each  team  member  must  have  credibility  with  senior 
management,  respect  within  the  organization,  and  the  ability  to  influence 
people. 

•  motivation  and  commitment.  Each  team  member  must  demonstrate  the 
motivation  to  improve  the  software  process,  the  commitment  to  act  as 
change  agent,  and  the  willingness  to  do  what  it  takes  to  achieve  assessment 
goals. 

Note  also  that  team  members  should  not  be  managers  of  one  of  the  selected  assessment 
projects  or  within  the  direct  supervisory  chain  of  any  of  the  anticipated  interviewees. 
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3.3  Who  Are  Appropriate  Assessment  Participants? 

Selecting  assessment  participants  includes  selection  of  projects  and  project  leaders,  ques¬ 
tionnaire  respondents,  and  functional  area  representative  (FAR)  interviewees. 

The  number  and  characteristics  of  selected  projects,  respondents,  and  interviewees  must  be 
a  representative  sample  of  the  appraised  entity. 

3.3.1  Selecting  Projects  and  Project  Leaders 

Projects  are  selected  for  the  assessment  where  detailed  investigation  will  be  focused.  If  a 
project  is  selected  for  the  assessment,  then  the  project  leader  and  selected  members  of  the 
project  staff  will  participate  in  project  leader  and  FAR  interviews.  Projects  should  be  selected 
in  such  a  manner  that  they  are  representative  of  the  organizational  entity  being  appraised. 

Factors  to  be  considered  in  project  selection  include  how  closely  the  selected  projects  repre¬ 
sent  variations  in  the  assessed  entity's  projects  in  the  following  areas:  application  type  and  do¬ 
main,  technology  employed,  and  scope  in  terms  of  product  size,  staff  size,  duration,  and  life- 
cycle  phases.  Normally  the  projects  selected  should  not  have  durations  less  than  six  months. 
Selected  projects  should  be  at  varying  points  in  the  life  cycle  with  emphasis  on  later  stages. 
In  addition,  project  selection  should  take  into  account  the  impact  of  the  project  on  the  business 
in  terms  of  revenue,  profit,  or  strategic  value. 

The  portion  of  the  appraised  entity  represented  by  the  projects  both  in  terms  of  number  of 
projects  and  number  of  staff  affect  the  confidence  of  the  results.  The  recommendation  is  to 
select  four  projects. 

The  goal  of  selecting  project  leader  interviewees  is  to  select  a  group  of  people  that  is  a  repre¬ 
sentative  sample  of  the  appraised  entity's  management  staff  at  the  project  level  and,  in  some 
instances,  below.  The  assumption  is  that  this  goal  is  achieved  by  selecting  projects  that  are 
representative  of  the  organization  and  interviewing  their  management  staff. 

Selected  project  leaders  must  be  committed  to  the  assessment  goals  and  the  software  pro¬ 
cess  improvement  initiative. 

Generally  the  project  leader  interviewees  will  include  individual  leaders  of  the  projects  select¬ 
ed  for  the  assessment.  However,  in  certain  cases,  the  project  leader  may  be  removed  from 
the  day-to-day  management  of  software  development  with  software  development  responsibil¬ 
ities  distributed  between  several  lower  level  managers.  In  such  cases,  the  project  leader  may 
be  selected  in  addition  to  a  combination  of  functional  managers,  such  as  the  software  manag¬ 
er,  the  configuration  manager,  the  test  manager,  etc. 

The  leader  of  each  project  is  interviewed  individually.  If  more  than  one  individual  from  the 
project’s  management  chain  is  involved  in  the  interview,  then  none  of  the  individuals  should 
be  in  the  reporting  chain  of  any  of  the  others. 
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3.3.2  Selecting  Questionnaire  Respondents 

Selection  of  maturity  questionnaire  respondents  must  be  balanced  against  the  value  of  the 
questionnaire  data  versus  the  interruption  to  the  organization.  The  questionnaire  will  be  used 
primarily  for  the  purpose  of  refining  the  data-gathering  strategy  for  the  on-site  activities.  Ques¬ 
tionnaire  data  will  be  corroborated  through  document  review  and  interviews;  it  is  not  the  sole 
basis  for  making  judgments  concerning  software  process  capability.  Given  these  facts,  wide¬ 
spread  administration  of  the  questionnaire  may  not  be  advantageous.  The  recommended 
number  of  questionnaire  respondents  is  between  4  and  10. 

Additional  factors  to  consider  in  selecting  respondents  include  the  extent  to  which  they  repre¬ 
sent  the  selected  project  leaders  and  lead  technical  staff.  At  a  minimum  the  project  leaders  of 
the  selected  projects  should  provide  answers  to  the  questionnaire.  In  addition,  key  technical 
personnel  or  functional  managers  for  the  organization  may  be  selected  to  answer  the  ques¬ 
tionnaire.  Inclusion  of  such  additional  respondents  may  be  of  value  in  organizations  where  the 
project  leaders  are  not  involved  in  day-to-day  software  development  activities  or  where  these 
activities  are  the  responsibility  of  multiple  functional  managers. 

3.3.3  Selecting  Functional  Area  Representatives  (FARs) 

The  goal  of  selecting  FAR  interviewees  is  to  select  a  group  of  people  that  forms  a  represen¬ 
tative  sample  of  the  appraised  entity’s  technical  staff.  FAR  interviewees  should  be  practitio¬ 
ners,  not  managers  or  staff.  They  should  include  opinion  leaders,  and  their  managers  should 
be  committed  to  software  process  improvement.  It  is  desirable  to  have  FARs  who  have  the 
interpersonal  skills  required  to  facilitate  a  free  flow  of  information  during  the  FAR  interview  ses¬ 
sions.  The  FARs  as  a  group  must  have  characteristics  or  attributes  (e.g.,  experience)  that 
closely  resemble  those  of  the  appraised  entity's  population.  The  FAR  interview  sessions 
should  include  persons  from  each  of  the  projects  that  were  selected  for  specific  investigation, 
as  well  as  persons  from  other  projects. 

No  two  individuals  who  have  a  reporting  relationship  to  each  other  should  be  in  a  FAR  inter¬ 
view  session  together.  Generally  each  FAR  session  should  have  four  to  eight  participants. 
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4  CBA  IPI  Method  Activities 


The  CBA  IPI  method  consists  of  three  phases.  The  first  phase  includes  the  activities  neces¬ 
sary  to  plan  and  prepare  for  the  assessment.  The  second  phase  consists  of  on-site  activities 
for  conducting  the  assessment,  including  techniques  for  gathering,  organizing,  and  consolidat¬ 
ing  data.  The  final  phase  is  to  report  the  results.  Each  phase  is  described  in  the  following  sec¬ 
tions. 

4.1  Plan  and  Prepare  the  Assessment 

This  section  describes  the  activities  that  take  place  before  the  on-site  period.  These  activities 
are  shown  in  Figure  2. 


Figure  2:  Pre-Onsite  Activities 


4.1 .1  Identify  Assessment  Scope 

The  purpose  of  this  activity  is  to  develop  a  common  understanding  between  the  assessment 
team  leader  and  the  sponsor  concerning  the  assessment  goals,  scope,  constraints,  roles,  re¬ 
sponsibilities,  and  outputs  and  to  obtain  commitments  to  proceed  with  the  assessment. 

4.1 .2  Develop  Assessment  Plan 

A  plan  for  conducting  the  assessment  is  developed  based  on  the  assessment  goals  identified 
by  the  sponsor.  Included  in  the  plan  are  detailed  schedules  for  the  assessment  as  well  as  any 
risks  identified  with  execution  of  the  assessment.  During  this  activity,  a  site  information  packet 
can  be  developed  that  will  assist  the  assessment  team  in  understanding  the  assessed  orga¬ 
nization.  Assessment  team  members,  projects,  and  assessment  participants  are  selected  ac¬ 
cording  to  defined  criteria.  Documents  are  identified  for  initial  review,  and  the  logistics  for  the 
on-site  visit  are  identified  and  planned. 
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4.1 .3  Prepare  and  Train  Team 

During  this  activity,  the  assessment  team  leader  must  ensure  that  each  assessment  team 
member  has  received  adequate  CMM  training  (at  least  equivalent  to  the  three-day  SEI  Intro¬ 
duction  to  the  CMM  course)  prior  to  being  trained  in  the  CBA I  PI  method.  The  team  leader  then 
conducts  the  three-day  CBA  IPI  Team  Training  so  that  each  team  member  understands  the 
assessment  process,  its  underlying  principles,  the  tasks  necessary  to  execute  it,  and  each 
person’s  role  in  performing  the  tasks.  Team  training  is  followed  by  planning  sessions  that  in¬ 
clude  establishing  ground  rules  for  the  assessment,  discussion  of  the  details  of  the  assess¬ 
ment,  and  the  scripting  of  interview  questions. 

4.1.4  Brief  Assessment  Participants 

The  purpose  of  this  activity  is  to  ensure  that  the  assessment  participants  understand  the  as¬ 
sessment  process  and  have  a  clear  set  of  expectations  regarding  its  outcomes. 

4.1.5  Administer  Questionnaire 

Information  about  the  organization’s  software  processes  is  collected  from  selected  members 
of  the  organization  using  the  SEI  maturity  questionnaire.  This  information  is  used  to  provide 
team  members  an  overview  of  the  organization’s  process  capability;  often,  comments  on  the 
questions  provide  more  value  to  the  team  than  the  yes/no  responses. 

4.1.6  Examine  Questionnaire  Responses 

The  assessment  team  members  examine  the  pattern  and  nature  of  responses  on  the  maturity 
questionnaires  completed  by  site  personnel.  Observations  are  not  made  based  on  question¬ 
naire  responses  alone.  Rather,  the  responses  provide  probing  points  for  later  activities  such 
as  interviews  and  document  reviews.  The  responses  assist  the  team  in  focusing  their  investi¬ 
gations. 

4.1.7  Conduct  Initial  Document  Review 

An  initial  set  of  documents  about  the  organization’s  software  processes  are  reviewed  to  find 
additional  areas  for  probing,  to  understand  the  life  cycle(s)  in  use  by  the  organization,  and  to 
map  organization  data  to  the  CMM. 

4.2  Conduct  the  Assessment 

This  section  describes  the  activities  that  take  place  during  the  on-site  period.  These  activities 
are  illustrated  in  Figure  3. 

4.2.1  Conduct  Opening  Meeting 

This  activity  kicks  off  the  on-site  visit  and  sets  participants’  expectations  concerning  the  as¬ 
sessment  activities.  The  sponsor  of  the  assessment  opens  the  presentation  to  show  visible 
support  and  urge  participants  to  be  forthcoming  in  interview  sessions. 
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4.2.2  Interviews 

The  purposes  of  interviewing  are  to  identify  areas  that  people  believe  can  and  should  be  im¬ 
proved  in  the  organization,  to  understand  how  the  work  is  performed,  to  understand  the  pro¬ 
cesses  in  use,  to  understand  the  relationships  among  the  organization-level  processes  and 
the  project-level  processes,  and  to  ensure  coverage  of  the  CMM  within  the  scope  of  the  as¬ 
sessment  across  the  organization,  as  defined  in  the  scope. 

•  Project  Jeaders  are  individually  interviewed  by  the  assessment  team. 

Typically,  four  representative  projects  are  selected,  and  their  project  leaders 
participate  in  these  in-depth  interviews.  These  interviews  generally  address 
issues  associated  with  project  management  and  the  processes  in  use  on  the 
project. 

•  Middle  managers  are  interviewed  as  a  group  to  understand  the  middle 
management  perspective  of  how  the  work  is  performed  in  the  organization, 
any  problem  areas  of  which  they  are  aware,  and  improvements  that  they  feel 
need  to  be  made. 

•  Functional  area  representatives  are  interviewed  to  collect  data  within  the 
scope  of  the  assessment  and  to  identify  areas  that  the  software  practitioners 
believe  can  and  should  be  Improved  in  the  organization.  Typical  functional 
area  representative  sessions  would  include  6-12  persons  grouped  by  areas 
such  as  software  engineering  process  group  (SEPG),  requirements,  design, 
code  and  unit  test,  integration,  and  system  test  activities. 


Day  1 


Day  N 


Figure  3:  Chronology  of  On-Site  Activities 
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4.2.3  Consolidate  Information 

The  purpose  of  this  activity  is  to  summarize  and  consolidate  information  into  a  manageable 
set  of  observations  which  are  categorized  with  reference  to  the  KPAs  of  the  CMM.  All  obser¬ 
vations  must  be  validated  using  the  rules  of  corroboration.  The  team  must  reach  consensus 
on  the  validity  of  observations  and  whether  sufficient  data  in  the  areas  being  investigated  have 
been  collected.  It  is  the  team’s  responsibility  to  obtain  sufficient  information  to  cover  the  orga¬ 
nization,  the  software  development  life  cycle,  and  the  CMM  components  within  the  assess¬ 
ment  scope  before  any  rating  can  be  done.  The  responsibilities  for  guiding  the  consolidation 
efforts  for  individual  KPAs  are  usually  divided  among  team  members. 

4.2.4  Prepare  Draft  Findings  Presentation 

Draft  findings  are  generated  from  the  observations  in  preparation  for  obtaining  feedback  from 
the  assessment  participants  who  have  provided  the  information  through  interviews.  Ratings 
are  not  considered  until  after  the  draft  findings  presentations,  as  the  assessment  team  is  still 
collecting  data. 

4.2.5  Present  Draft  Findings 

The  purpose  of  this  activity  is  to  obtain  feedback  from  the  assessment  participants  on  the  as¬ 
sessment  team’s  draft  findings.  Feedback  is  recorded  for  the  team  to  consider  at  the  conclu¬ 
sion  of  all  of  the  draft  findings  presentations.  Draft  findings  are  presented  in  multiple  sessions 
in  order  to  protect  the  confidentiality  of  the  assessment  participants. 

4.2.6  Consolidate,  Rate,  and  Prepare  Final  Findings 

The  assessment  team  consolidates  additional  data  obtained  during  the  draft  findings  presen¬ 
tations  and  any  follow-up  interviews  and  document  review.  When  the  team  has  achieved  full 
coverage  of  the  CMM,  the  organization,  and  the  software  life  cycle,  the  rating  process  may 
begin  by  rating  each  goal  for  each  key  process  area  (KPA)  within  the  assessment  scope.  Rat¬ 
ings  are  always  established  based  on  consensus  of  the  entire  assessment  team.  For  each 
goal,  the  team  reviews  all  weaknesses  that  relate  to  that  goal  and  asks:  “Is  there  a  weakness 
significant  enough  to  have  a  negative  impact  on  the  goal?”  If  so,  the  goal  is  rated  unsatisfied. 
If  the  team  decides  that  there  are  no  significant  weaknesses  that  have  an  impact  on  a  goal,  it 
is  rated  satisfied.  For  a  KPA  to  be  rated  satisfied,  all  goals  for  the  KPA  must  be  rated  satisfied. 
A  KPA  may  be  rated  satisfied,  unsatisfied,  not  applicable,  or  not  rated.  Assignment  of  a  matu¬ 
rity  level  rating  is  optional  at  the  discretion  of  the  sponsor.  For  a  particular  maturity  level  rating 
to  be  achieved,  all  key  process  areas  within  and  below  a  given  maturity  level  must  be  satisfied. 
For  example,  for  an  organization  to  be  rated  at  maturity  level  3,  all  KPAs  at  level  3  and  at  level 
2  must  have  been  investigated  during  the  assessment,  and  all  KPAs  must  have  been  rated 
satisfied  by  the  assessment  team.  The  final  findings  presentation  is  developed  by  the  team  to 
present  to  the  sponsor  and  the  organization  the  strengths  and  weaknesses  observed  for  each 
KPA  within  the  assessment  scope,  the  ratings  of  each  KPA,  and  the  maturity  level  rating  if  de¬ 
sired  by  sponsor. 
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4.3  Report  Results 


4.3.1  Present  Final  Findings 

The  team  leader,  or  designated  presenter,  presents  the  assessment  results  to  the  sponsor. 
The  sponsor  owns  the  assessment  results  and  is  free  to  use  them  as  he  or  she  sees  fit.  During 
the  final  findings  presentation,  the  presenter  must  ensure  that  the  organization  understands 
the  issues  that  were  discovered  during  the  assessment  and  the  key  software  process  issues 
that  it  faces.  Organizational  strengths  are  presented  to  validate  what  the  organization  is  doing 
well.  Strengths  and  weaknesses  are  presented  for  each  key  process  area  within  the  assess¬ 
ment  scope  as  well  as  any  non-CMM  issues  that  affect  process.  A  KPA  profile  is  presented 
showing  the  individual  KPA  ratings. 

4.3.2  Conduct  Executive  Session 

The  purpose  of  this  activity  is  to  allow  the  senior  site  manager  to  clarify  any  issues  with  the 
assessment  team,  to  confirm  his  or  her  understanding  of  the  software  process  issues,  and  to 
get  guidance  regarding  the  focus,  timing,  and  priorities  of  the  recommendations  report  and  fol¬ 
low-on  activities. 

4.3.3  Wrap-Up  of  the  Assessment 

The  team  leader  collects  feedback  from  the  assessment  participants  and  the  assessment 
team  on  the  assessment  process,  collects  information  that  needs  to  be  reported  to  the  SEI, 
and  assigns  responsibilities  for  follow-on  activities. 
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Time  Frame  and  Resource  Requirements 


In  order  to  identify  the  resources  that  are  needed  to  perform  an  assessment,  a  typical  time 
frame  along  with  people  required  are  discussed  in  the  following  subsections. 

5.1  Assessment  Time  Frame 

A  typical  assessment  time  frame  is  shown  in  Table  1 . 


Months  1-2 

Assessment  planning  and  team  training 

Month  3 

On-site  assessment 

Month  4 

Final  report  delivery  and  recommendations 
briefing 

Month  5 

Action  pian  development 

Months  6-24 

Action  plan  implementation 

Months  1 8-30 

Re-assessment 

Table  1 :  Typical  Assessment  Time  Frame 
5.2  Resource  Requirements 

Resources  required  for  an  organization  to  perform  a  CBA  IPI  vary  according  to  the  scope  of 
the  assessment.  The  numbers  in  Table  2  may  be  used  as  guidelines  in  planning  the  resource 
requirements  for  an  assessment.  These  figures  were  gathered  from  early  implementations  of 
CBA  IPI  assessments.  However,  subsequent  assessments  have  shown  that  the  assessments 
can  be  done  more  efficiently,  so  we  consider  these  estimates  to  be  conservative  and  on  the 
high  end.  For  a  typical  assessment  team  consisting  of  8  team  members  plus  the  assessment 
team  leader,  investigating  4  representative  projects,  and  interviewing  40  functional  area  rep¬ 
resentatives,  a  CBA  IPI  is  estimated  to  require  approximately  200  person-days,  beginning  with 
assessment  planning  through  writing  the  final  report  and  hand-off  to  the  team  who  will  plan  the 
actions  for  continuing  process  improvement  activities. 
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Planning 

Assessment  team  leader 

Sponsor 

Site  coordinator 

1 0-20  days 

3-5  days 

1 0-20  days 

Pre-Onsite  Activities 

-  Assessment  team  leader 

Site  coordinator 

Assessment  team  members 

10-14  days 

10-14  days 

6-8  days  each 

On-Site  Activities 

Assessment  team  leader 

Sponsor 

Site  coordinator 

Assessment  team  members 

Assessment  participants  (project 
leaders,  middle  managers,  and 
functional  area  representatives) 

6-10  days 

4- 5  hours 

6-10  days 

5- 10  days  each 

1 .5  days  each 

Post-Assessment 

Activities 

Assessment  team  leader 

Sponsor 

Site  coordinator 

Assessment  team  members 

4-8  days 

1  day 

2  days 

1  -2  days  each 

Totals 

Assessment  team  leader 

Sponsor 

Site  coordinator 

Assessment  team  members 

Assessment  participants  (project 
leaders,  middle  managers,  and 
functional  area  representatives) 

30-52  days 

8-11  days 

28-46  days 

1 2-20  days  each 

1 .5  days  each 

Table  2:  Resource  Requirements  for  CBA IPI 
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6  Foliow-On  Activities 


To  benefit  from  the  assessment,  the  organization  must  have  an  improvement  infrastructure  in 
place.  This  infrastructure  would  be  in  the  form  of  sponsorship,  establishment  of  a  software  en¬ 
gineering  process  group  (SEPG),  CMM  training  for  the  organization,  etc. 

The  assessment  team  will  develop  recommendations  and  document  the  assessment  results 
immediately  following  the  final  findings  briefing.  Although  a  final  report  is  optional  but  recom¬ 
mended,  if  a  final  report  is  not  required  by  the  sponsor,  one  of  the  team  members  should  be 
responsible  for  collecting  the  assessment  details,  removing  all  attribution,  and  making  them 
available  to  the  action  planning  team.  These  assessment  details  are  critical  for  prioritizing  is¬ 
sues,  developing  the  action  plan,  and  implementing  the  establishing  phase.  There  are  several 
benefits  to  the  establishing  phase:  it  provides  an  organized  and  planned  approach  to  software 
process  improvement;  a  clear  understanding  of  costs,  schedules,  and  resources;  and  the  op¬ 
portunity  to  plan  for  actions  that  the  organization  has  the  capability  to  achieve.  It  is  important 
to  move  into  the  establishing  phase  as  soon  as  possible  after  completion  of  the  assessment 
so  that  momentum  from  the  assessment  can  be  maintained  and  the  staff’s  expectations  can 
be  managed.  An  assessment  provides  expectations  for  improvement  in  an  organization;  fol¬ 
low-on  activities  are  very  important  to  avoid  disappointment  or  disillusionment  by  the  assess¬ 
ment  participants. 
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7  Future  Improvement  of  the  Method 


There  are  many  feedback  mechanisms  in  place  to  enable  the  SEI  to  continuously  improve  the 
method.  The  SEI  requires  Lead  Assessors  to  provide  feedback  following  each  assessment 
that  they  lead.  In  addition,  each  sponsor  and  each  assessment  team  member  is  asked  to  pro¬ 
vide  feedback  to  the  SEI  at  the  conclusion  of  an  assessment.  Change  requests  are  accepted 
by  the  SEI  from  anyone  who  wishes  to  send  one  in. 

There  is  no  major  change  anticipated  to  this  assessment  method  until  after  CMM  V2.0  is  re- 
ieased.  However,  minor  modifications  wili  be  made  from  time  to  time  through  notices  to  the 
Lead  Assessors. 
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Appendix  A  SEI  Appraiser  Program 


SEI  Appraiser  Program 

The  SEI  Appraiser  Program  is  designed  to  maintain  the  quaiity  of  participants  in  appraisal 
technology  within  the  software  community.  The  goals  of  the  program  are 

•  to  maximize  the  value  and  use  of  SEI  appraisal  methods,  designed  and 
facilitated  by  qualified,  trained  individuals,  as  part  of  a  systematic 
improvement  program  within  organizations  that  produce  software 

•  to  transition  appraisal  technology  to  SEI  clients  in  an  effective  manner, 
maintaining  consistency  and  quality  in  the  process. 

The  SEI  strives  to  ensure  the  continued  confidence  of  the  software  community  in  the  quality 
of  SEI  process  appraisal  technologies.  The  SEI  Appraiser  Program  selects  and  trains  the 
highest  quality  candidates  to  lead  appraisals.  Persons  meeting  the  requirements  of  the  pro¬ 
gram  have  credentials  that  distinguish  them.  They  have  access  to  SEI  appraisal  methods, 
training  materials,  technical  support,  and  upgrade  training.  Through  their  participation  in  ap¬ 
praisals  and  through  feedback  mechanisms  built  into  appraisal  methodologies,  they  partici¬ 
pate  in  the  advancement  of  appraisal  technologies.  The  Appraiser  Program  is  intended  in  time 
to  encompass  multiple  appraisal  methods,  specifically,  assessments  and  evaluations.  Lead 
Assessors  are  authorized  within  the  program  to  perform  assessments.  Lead  Evaluators  may 
become  authorized  in  the  future  to  conduct  evaluations. 

Lead  Assessors 

Lead  Assessors  are  authorized  to  market  and  perform  assessments  either  for  third-party  or¬ 
ganizations  or  for  their  own  organizations’  internal  use.  Current  Lead  Assessors  are  trained  in 
the  CMM-Based  Appraisal  for  Internal  Process  Improvement  (CBA  IPI).  The  CBA  IPI  method 
is  an  SEI  product  that  provides  the  software  community  a  reliable  assessment  of  an  organiza¬ 
tion’s  software  process  and  provides  guidance  for  improving  the  process. 

Lead  Assessors  commit  to 

•  sign  a  letter  of  agreement  with  the  SEI  that  identifies  Lead  Assessor 
responsibilities 

•  verify  that  assessment  team  members  have  met  the  CMM  training 
requirement 

•  conduct  CBA  IPI  Team  Training  for  assessment  teams.  Training  teams  to 
conduct  independent  assessments  without  a  Lead  Assessor  is  not  permitted. 
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•  lead  or  participate  in  at  least  two  CBA IPI  assessments  within  each  24-month 
authorization  period  and,  for  each  completed  assessment,  file  a  complete 
assessment  report  with  the  SEI 

•  obtain  and  use  SEI  copyrighted  materials  for  CBA  assessments 

•  use  a  new  Lead  Assessor’s  kit  for  each  assessment  or  purchase  a  Lead 
Assessor  multi-assessment  kit  according  to  a  quantity  price  (with  permission 
to  reproduce  these  materials  for  a  designated  period  of  time) 

•  complete  successfully  any  required  upgrade  courses  and  examinations,  and 
accept  and  use  upgraded  materials  when  available 

•  cooperate  in  random  audits  of  assessments  by  the  SEI  and  take  any 
recommended  remedial  action  as  a  result  of  the  audit 

Lead  Assessors  are  encouraged  to 

•  present  educational  assessment  awareness  classes 

•  observe  candidate  Lead  Assessors  and  complete  observations  reports 

Becoming  a  Lead  Assessor 

To  participate  in  the  SEI  Appraiser  Program  and  become  an  authorized  Lead  Assessor,  appli¬ 
cants  must  meet  certain  prerequisites: 

•  verif  ied  participation  as  an  assessment  team  member  in  at  least  2  CBA  I  Pis 
within  the  prior  24-month  period  (SEI  verifies  from  the  Process  Appraisal 
Information  System) 

•  minimum  of  10  years  of  software  engineering  experience  in  software 
development  or  maintenance  in  an  appropriate  technical  area,  e.g.,  software 
design,  software  quality  assurance,  requirements  analysis,  configuration 
management,  and  testing 

•  minimum  of  2  years  of  experience  managing  software  development 
personnel 

•  advanced  degree  or  equivalent  experience  in  an  appropriate  technical  area 

•  successful  completion  of  the  SEI  course  Introduction  to  the  CMM. 

It  is  also  recommended  that  applicants  have 

•  oral  and  written  communication  skills 

•  the  ability  to  interact  with  management  and  members  of  the  technical  staff 

•  demonstrated  knowledge  and  appreciation  of  software  process 

•  good  technical  and  instructional  presentation  training  skills 

•  the  ability  to  work  effectively  in  a  team  environment  and  knowledge  of  group 
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facilitation  and  team-building  techniques 

•  knowledge  of  total  quality  management  and  quality  improvement  techniques. 

Once  the  prerequisites  are  met,  applicants  must  submit  a  SEI  Appraiser  Program  application, 
a  software  process  background  resume,  and  a  registration  form  to  attend  an  offering  of  CBA 
Lead  Assessor  Training  at  the  SEI.  At  the  successful  completion  of  training,  the  applicant  is 
considered  to  be  a  candidate  Lead  Assessor. 

A  candidate  Lead  Assessor  must  perform  as  a  CBA  IPI  team  leader,  under  the  observation  of 
an  authorized  Lead  Assessor,  within  24  months  after  successful  completion  of  training.  A  can¬ 
didate  Lead  Assessor  must  receive  a  satisfactory  observation  report  and  recommendation 
from  the  observing  Lead  Assessor  to  become  fully  authorized. 

Authorization  Terms  for  Lead  Assessors 

Authorization  as  a  Lead  Assessor  is  valid  for  a  two-year  period.  Authorization  must  be  re¬ 
newed  and  a  renewal  fee  paid  every  two  years.  Renewal  is  not  automatic.  Lead  Assessors 
must  satisfactorily  fulfill  their  responsibilities  during  the  previous  two  years  to  be  renewed  for 
another  two-year  period. 

The  following  factors  are  considered  during  the  renewal  process; 

•  reports  from  audits  that  may  have  taken  place  during  the  authorization  period 

•  reports  from  assessment  clients  returned  to  the  SEI  during  the  authorization 
period 

•  review  of  the  actual  assessment  data  to  determine  that  the  prescribed 
process  was  followed. 

Contact  Information 

If  you  have  questions  or  would  like  more  information  about  the  SEI  Appraiser  Program,  con¬ 
tact: 

Customer  Relations 
Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213-3890 
Phone 

412/268-5800 

Internet;  customer-relations@sei.cmu.edu 
World  Wide  Web:  http://www.sei.cmu.edu 
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Appendix  B  Method  Roles  and  Responsibilities 


There  are  a  number  of  roles  associated  with  the  various  assessment  tasks.  Individuals  may 
assume  multiple  roles  during  the  assessment,  and  roles  may  rotate  when  appropriate. 

Assessment  team  leader.  Experienced  individual  who  leads  the  assessment  and  keeps  the 
team  together  and  on  track.  The  assessment  team  leader  must  be  an  authorized  Lead  Asses¬ 
sor  in  the  SEI  Appraiser  Program.  During  the  assessment  planning  phase  the  team  leader  is 
responsible  for  soliciting  sponsor  input,  explaining  the  impact  of  the  assessment  scope  and 
constraints  on  the  assessment  goals,  providing  cost/schedule  estimates,  setting  expectations, 
and  obtaining  a  commitment  to  proceed. 

The  team  leader  is  also  responsible  for  ensuring  that  all  pre-onsite  planning  activities  have 
been  completed  and  all  preparation  for  the  assessment  is  complete,  including  training  the  as¬ 
sessment  team.  The  team  leader  may  have  assistance  from  a  qualified  person  during  presen¬ 
tation  of  the  team  training,  although  the  team  leader  has  the  responsibility  to  see  that  team 
training  is  performed  successfully.  During  the  conduct  of  the  assessment,  the  team  leader  as¬ 
signs  responsibilities  for  assessment  tasks,  facilitates  interviews,  monitors  team  member  and 
interviewee  performance,  acts  as  timekeeper  (or  assigns  one),  and  manages  adherence  to 
the  assessment  process  and  schedule. 

Assessment  team  members.  Assessment  team  members  are  required  to  have  taken  the  SEI 
Introduction  to  CMM,  or  an  equivalent  course  in  the  team  leader’s  judgment,  and  received 
CBA  IPI  Team  Training  from  the  assessment  team  leader.  Each  team  member  is  responsible 
for  reviewing  the  site  information  packet  and  identifying  documents  for  initial  review.  Each 
team  member  is  also  responsible  for  asking  questions  during  interview  sessions,  reviewing 
notes,  identifying  and  classifying  significant  information  obtained  during  interviews  and  docu¬ 
ment  review,  and  identifying  additional  information  required.  Team  members  are  responsible 
for  coming  to  consensus  on  the  assessment  findings  and  ratings. 

Functional  area  representatives  (FARs).  Site  representatives  who  are  practitioners  and 
have  technical  responsibilities  for  certain  domains.  The  FARs  are  responsible  for  attending  the 
assessment  participants’  briefing,  opening  meeting,  the  FAR  interview  session  to  which  they 
are  assigned,  the  draft  findings  presentation,  and  the  final  findings  presentation. 

Middle  managers.  Site  representatives  who  fall  between  the  project  leaders  and  the  senior 
site  manager  in  the  organizational  hierarchy.  The  middle  managers  are  responsible  for  attend¬ 
ing  the  assessment  participants’  briefing,  the  opening  meeting,  the  middle  manager  interview 
session  to  which  they  are  assigned,  the  draft  findings  presentation,  and  the  final  findings  pre¬ 
sentation. 


37 


CMU/SEI-96-TR-007 


Project  leaders.  Site  representatives  who  have  the  leadership  responsibility  for  the  software 
aspect  of  one  of  the  projects  selected  for  the  assessment.  The  project  leaders  are  responsible 
for  attending  the  assessment  participant’s  briefing,  opening  meeting,  the  project  leader’s  in¬ 
terview  session  to  which  they  are  assigned,  the  draft  findings  presentation,  and  the  final  find¬ 
ings  presentation. 

Questionnaire  respondents.  Site  representatives  who  are  selected  to  complete  the  maturity 
questionnaire.  Usually  the  project  leaders  who. have  been  selected  for  individual  interviews 
are  asked  to  complete  the  questionnaire,  and  there  may  be  additional  questionnaire  respon¬ 
dents  from  the  organization. 

Senior  site  manager  or  Sponsor.  The  person  who  acts  as  sponsor  of  the  assessment  and 
gives  the  team  leader  commitment  to  proceed  in  the  assessment  start-up  phase.  The  sponsor 

•  identifies  to  the  team  leader  the  business  goals  that  bear  on  the 
organization’s  software  development  and  maintenance  activity 

•  describes  to  the  team  leader  any  current  quality  improvement  initiatives  in 
the  appraised  organization 

•  identifies  the  scope  of  the  assessment  and  any  constraints  that  will  exist 

•  provides  authorization  to  proceed  and  personally  participates  in  the  opening 
meeting  by  urging  participants  to  be  forthcoming  and  supportive  of  the 
assessment  effort 

•  is  responsible  for  selecting  the  assessment  team  with  selection  criteria  and 
input  provided  by  the  assessment  team  leader  (This  task  may  be  delegated 
to  someone  very  familiar  with  the  organization’s  staff,  the  organization’s 
software  process,  and  the  assessment  process.) 

•  is  responsible  for  approving  the  assessment  plan 

•  is  the  recipient  of  the  final  findings  presentation  and  is  responsible  for 
providing  resources  for  follow-on  process  improvement  activities 

There  may  be  circumstances  where  the  sponsor  of  the  assessment  represents  a  corporate 
process  improvement  organization,  and  the  senior  site  manager  is  in  charge  of  the  organiza¬ 
tion  being  assessed.  In  that  instance,  the  team  leader  must  clarify  the  roles  between  the  spon¬ 
sor  and  the  senior  site  manager,  and  both  must  app.rove  the  assessment  plan. 

Site  coordinator.  The  individual  responsible  for  handling  the  logistics  of  the  assessment.  The 
site  coordinator  is  responsible  for  developing  the  schedule,  notifying  the  assessment  partici¬ 
pants  of  the  schedule,  making  sure  that  adequate  rooms  have  been  reserved  for  both  the  pre- 
onsite  and  onsite  periods,  making  and  distributing  copies  of  the  schedules,  making  sure  that 
all  necessary  supplies  and  equipment  are  available  when  needed,  scheduling  contingency  in¬ 
terviews,  requesting  additional  documentation,  and  ensuring  that  meals  are  taken  care  of.  The 
site  coordinator  needs  to  be  a  member  of  the  assessment  team. 
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Appendix  C  Glossary 


appraisal  A  diagnostic  performed  by  a  trained  team  to  evaluate  aspects  of 

an  organization’s  software  development  process,  e.g.,  CMM- 
Based  Appraisal  for  Internal  Process  Improvement  (CBA  IPI), 
Software  Capability  Evaluation  (SCE). 

assessed  entity  The  organizational  units  to  which  assessment  outputs  apply.  An 
assessed  entity  may  be  any  portion  of  an  organization  including 
an  entire  company,  a  selected  business  unit,  a  specific  geo¬ 
graphic  site,  units  supporting  a  particular  product  line,  units  in¬ 
volved  in  a  particular  type  of  service,  an  individual  project,  or  a 
multi-company  team. 

assessment  An  appraisal  by  a  trained  team  of  software  professionals  to  de¬ 

termine  the  state  of  an  organization’s  current  software  process, 
to  determine  the  high-priority  software  process- related  issues 
facing  an  organization,  and  to  obtain  the  organizational  support 
for  software  process  improvement,  e.g..  Software  Process  As¬ 
sessment  (SPA),  CMM-Based  Appraisal  for  Internal  Process 
Improvement  (CBA  IPI). 

assessment  goals  The  desired  outcome  of  an  assessment  process. 

assessment  scope  The  organizational  entities  and  CMM  components  selected  for 

investigation. 

assessment  sponsor  The  individual  who  authorizes  an  assessment,  defines  its  goals 
and  constraints,  and  commits  to  the  use  of  the  assessment  out¬ 
puts. 

CAF  compliant  appraisal  method 

An  appraisal  method  that  conforms  to  appraisal  method  require¬ 
ments  defined  in  the  CMM  Appraisal  Framework  (CAF)  [Mas¬ 
ters  95]. 

Capability  Maturity  Mode^^  for  Software(CMM) 

A  description  of  the  stages  through  which  software  organiza¬ 
tions  evolve  as  they  define,  implement,  measure,  control,  and 
improve  their  software  processes.  This  model  provides  a  guide 
for  selecting  process  improvement  strategies  by  facilitating  the 
determination  of  current  process  capabilities  and  the  identifica- 
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tion  of  the  issues  most  critical  to  software  quality  and  process 
improvement.  CMM  Version  1 .1  is  specified  in  CMU/SEI-93-TR- 
24  and  CMU/SEI-93-TR-25. 

CMM  Appraisal  Framework  (CAF) 

A  framework  for  planning,  conducting,  and  completing  CMM- 
based  appraisals. 

CMM-Based  Appraisal  for  Internal  Process  Improvement  (CBA  I  PI) 

An  assessment  developed  at  the  SEI  to  determine  an  organiza¬ 
tion’s  current  state  of  the  software  development  process  in  or¬ 
der  to  further  the  organization’s  own  internal  software  process 
improvement  program.  CBA  IPIs  are  based  on  the  CMM  for 
Software  V1 .1  and  comply  with  the  CAF. 

CMM  scope  of  the  assessment 

The  portion  of  the  CMM  used  as  a  framework  for  evaluating  an 
organization's  software  development  process  during  an  assess¬ 
ment. 


common  feature  The  subdivision  categories  of  the  CMM  key  process  areas.  The 
common  features  are  attributes  that  indicate  whether  the  imple¬ 
mentation  and  institutionalization  of  a  key  process  area  is  effec¬ 
tive,  repeatable,  and  lasting.  The  CMM  common  features  are 
the  following: 


confidentiality 


consensus 


consolidation 


•  commitment  to  perform 

•  ability  to  perform 

•  activities  performed 

•  measurement  and  analysis 

•  verifying  implementation 

An  agreement  by  which  data  will  not  be  attributed  to  a  particular 
individual,  project,  or  organization. 

A  method  of  decision  making  that  allows  team  members  to  de¬ 
velop  a  common  basis  of  understanding  and  develop  general 
agreement  concerning  a  decision. 

The  activity  of  collecting  and  summarizing  the  information  pro¬ 
vided  into  a  manageable  set  of  data,  to  determine  the  extent  to 
which  the  data  are  corroborated  and  cover  the  areas  being  in¬ 
vestigated,  to  determine  the  data’s  sufficiency  for  making  judg- 
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merits,  and  to  revise  the  data  gathering  plan  as  necessary  to 
achieve  this  sufficiency.  This  activity  is  repeated  daily  following 
data  collection  activities  during  the  on-site  period. 

corroboration  Confirmation:  all  assessment  observations  must  be  confirmed 

by  information  from  different  sources  and  different  data-gather- 
ing  sessions  prior  to  use  as  findings. 

coverage  The  extent  to  which  data  gathered  addresses  CMM  compo¬ 

nents,  organizational  units,  and  life-cycle  phases  within  the 
scope  of  an  assessment. 

coverage  criteria  A  CMM  component  is  considered  to  be  covered  if  the  data  gath¬ 
ered  relevant  to  the  component 

•  are  representative  of  the  organizational  units  within  the  scope  of 
the  assessment 

•  are  representative  of  the  life-cycle  phases  within  the  scope  of  the 
assessment 

•  address  each  of  the  key  practices  of  the  activities  performed  and 
the  institutionalization  common  features  in  enough  depth  to  deter¬ 
mine  the  extent  of  their  implementation,  in  the  collective  opinion  of 
the  assessment  team 

document  A  collection  of  data,  regardless  of  the  medium  on  which  it  is  re¬ 

corded,  that  generally  has  permanence  and  can  be  read  by  hu¬ 
mans  or  machines. 

findings  The  conclusions  of  an  assessment,  evaluation,  audit,  or  review 

that  identify  the  most  important  issues,  problems,  or  opportuni¬ 
ties  within  the  area  of  investigation. 

functional  area  representatives  (FARs) 

The  assessment  site  representatives  who  are  practitioners  and 
have  technical  responsibilities  for  certain  domains  within  the 
software  development  process. 

goat  A  summary  of  the  key  practices  of  a  key  process  area  that  can 

be  used  to  determine  whether  an  organization  or  project  has  ef- 
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fectively  implemented  the  key  process  area.  The  goals  signify 
the  scope,  boundaries,  and  intent  of  each  key  process  area. 

IDEAL  approach  A  life-cycle  approach  for  process  improvement.  IDEAL  stands 
for  the  five  phases  of  the  approach:  Initiating,  Diagnosing,  Es¬ 
tablishing,  Acting,  and  Leveraging. 

institutionalization  The  building  of  infrastructure  and  corporate  culture  that  support 
methods,  practices,  and  procedures  so  that  they  are  the  ongo¬ 
ing  way  of  doing  business,  even  after  those  who  originally  de¬ 
fined  them  are  gone. 

key  practices  The  infrastructures  and  activities  that  contribute  most  to  the  ef¬ 

fective  implementation  and  institutionalization  of  a  key  process 
area. 

key  process  area  (KPA) 

A  cluster  of  related  activities  that,  when  performed  collectively, 
achieve  a  set  of  goals  considered  important  for  establishing  pro¬ 
cess  capability.  The  key  process  areas  have  been  identified  by 
the  SEI  to  be  the  principal  building  blocks  to  help  determine  the 
software  process  capability  of  an  organization  and  understand 
the  improvements  needed  to  advance  to  higher  maturity  levels. 

maturity  level  A  well-defined  evolutionary  plateau  toward  achieving  a  mature 

software  process.  The  five  maturity  levels  in  the  SEI's  Capability 
Maturity  Model  are  initial,  repeatable,  defined,  managed,  and 
optimizing. 

maturity  questionnaire  (MQ) 

A  set  of  questions  about  the  software  process  that  sample  the 
key  practices  in  each  key  process  area  of  the  CMM.  The  matu¬ 
rity  questionnaire  is  used  as  a  springboard  to  appraise  the  ca¬ 
pability  of  an  organization  or  project  to  execute  a  software  pro¬ 
cess  reliably. 

organization  A  unit  within  a  company  or  other  entity  (e.g.,  government  agen¬ 

cy  or  branch  of  service)  within  which  many  projects  are  man¬ 
aged  as  a  whole.  All  projects  within  an  organization  share  a 
common  top-level  manager  and  common  policies. 

process  A  sequence  of  steps  performed  for  a  given  purpose;  for  exam¬ 

ple,  the  software  development  process.  [IEEE  90].  A  set  of  ac- 
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tivities,  methods,  and  practices  that  guide  people  (with  their 
software  tools)  in  the  production  of  software. 

process  maturity  The  extent  to  which  a  specific  process  is  explicitly  defined,  man¬ 
aged,  measured,  controlled,  and  effective.  Maturity  implies  a 
potential  for  growth  in  capability  and  indicates  both  the  richness 
of  an  organization's  software  process  and  the  consistency  with 
which  it  is  applied  in  projects  throughout  the  organization. 

process  measurement 

The  set  of  definitions,  methods,  and  activities  used  to  take  mea¬ 
surements  of  a  process  and  its  resulting  products  for  the  pur¬ 
pose  of  characterizing  and  understanding  the  process. 

project  An  undertaking  requiring  concerted  effort,  which  is  focused  on 

developing  and/or  maintaining  a  specific  product.  The  product 
may  include  hardware,  software,  and  other  components.  Typi¬ 
cally  a  project  has  its  own  funding,  cost  accounting,  and  delivery 
schedule. 

rating  A  characterization  of  an  organization's  software  process  relative 

to  a  component  of  the  CMM. 

requirement  (1 )  A  condition  or  capability  needed  by  a  user  to  solve  a  problem 

or  achieve  an  objective.  (2)  A  condition  or  capability  that  must 
be  met  or  possessed  by  a  system  or  system  component  to  sat¬ 
isfy  a  contract,  standard,  specification,  or  other  formally  im¬ 
posed  documents.  (3)  A  documented  representation  of  a  condi¬ 
tion  or  capability  in  (1)  or  (2).  [IEEE  90] 

satisfied  Rating  given  to  a  CMM  component  that  is  applicable  in  an  orga¬ 

nization's  business  environment  and  is  performed  either  as  de¬ 
fined  in  the  CMM  or  with  an  adequate  alternative. 

site  A  geographic  location  of  one  or  more  of  an  organization's  units. 
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